Unskips visualize integration tests #142917

TinaHeiligers · 2022-10-06T20:33:38Z

Summary:
Unskips tests as these pass locally as-is without any changes to see if post-8.0 resolved the flakiness.

Note: 1/50 flaky test runs failed. Still WIP for now.
Update: Attempt2
Flaky test run succeeded 50/50 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1428

elasticmachine · 2022-10-06T20:33:41Z

Pinging @elastic/kibana-core (Team:Core)

TinaHeiligers · 2022-10-13T22:54:58Z

test/functional/page_objects/common_page.ts

@@ -467,7 +467,7 @@ export class CommonPageObject extends FtrService {
  async waitForSaveModalToClose() {
    this.log.debug('Waiting for save modal to close');
    await this.retry.try(async () => {
-      if (await this.testSubjects.exists('savedObjectSaveModal')) {
+      if (await this.testSubjects.exists('savedObjectSaveModal', { timeout: 5000 })) {


increase timeout

TinaHeiligers · 2022-10-14T00:59:13Z

@elasticmachine merge upstream

TinaHeiligers

self review

TinaHeiligers · 2022-10-13T22:56:25Z

test/functional/services/listing_table.ts

@@ -51,6 +51,16 @@ export class ListingTableService extends FtrService {
    return visualizationNames;
  }

+  private async getAllSelectableItemsNamesOnCurrentPage(): Promise<string[]> {
+    const visualizationNames = [];
+    const links = await this.find.allByCssSelector('.euiTableRow-isSelectable .euiLink');


.euiTableRow .euiLink matches on an empty table. I chose to add a new method rather than risk affecting other tests that implement the method.

TinaHeiligers · 2022-10-13T22:57:34Z

test/functional/services/listing_table.ts

+   * Navigates through all pages on Landing page and returns array of items names that are selectable
+   * Added for visualize_integration saved object tagging tests
+   */
+  public async getAllSelectableItemsNames(): Promise<string[]> {


Implements selecting "selectable" items

TinaHeiligers · 2022-10-13T22:58:43Z

x-pack/test/saved_object_tagging/common/fixtures/es_archiver/visualize/data.json

-
+},
+{
+  "id": "myextratag",


Tag for adding to a visualization that's been edited

TinaHeiligers · 2022-10-13T22:59:36Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts

-  // Failing: See https://github.com/elastic/kibana/issues/89958
-  describe.skip('visualize integration', () => {
+    await testSubjects.click('confirmSaveSavedObjectButton');
+    await retry.waitForWithTimeout('Save modal to disappear', 5000, () =>


Increased timeout waiting for the modal to clear

TinaHeiligers · 2022-10-13T23:01:12Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts

    before(async () => {
+      // clean up any left-over visualizations and tags from tests that didn't clean up after themselves
+      await kibanaServer.savedObjects.clean({ types: ['tag', 'visualization'] });


I found that if previous tests hadn't cleaned up after themselves, we get a mismatch on the visualizations remaining.

TinaHeiligers · 2022-10-13T23:02:02Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts

@@ -77,48 +115,46 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
      it('allows to manually type tag filter query', async () => {
        await listingTable.searchForItemWithName('tag:(tag-1)', { escape: false });
        await listingTable.expectItemsCount('visualize', 2);
-        const itemNames = await listingTable.getAllItemsNames();
+        const itemNames = await listingTable.getAllSelectableItemsNames();


Implements selecting all visualizations by 'euiTableRow-isSelectable`

TinaHeiligers · 2022-10-13T23:02:45Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts

-
-        await testSubjects.click('confirmSaveSavedObjectButton');
-        await PageObjects.common.waitForSaveModalToClose();
+        await createSimpleMarkdownVis({


Uses helper method

TinaHeiligers · 2022-10-13T23:04:45Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts


        await PageObjects.visualize.gotoVisualizationLandingPage();
        await listingTable.waitUntilTableIsLoaded();

-        await selectFilterTags('tag-1');
-        const itemNames = await listingTable.getAllItemsNames();
+        await selectFilterTags('myextratag');


We cannot guarantee the order in which the tests in this suite will run or if any are skipped again in the future, so we use a different tag to add to an existing visualization.

TinaHeiligers · 2022-10-13T23:06:09Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts

        await PageObjects.visualize.gotoVisualizationLandingPage();
-        await listingTable.waitUntilTableIsLoaded();
+        await PageObjects.visualize.deleteAllVisualizations();


Finding all links by css-matchers takes time and causes timeouts. We clean up all the other visualizations to increase rendering speed.

kibana-ci · 2022-10-14T01:55:36Z

💚 Build Succeeded

Buildkite Build
Commit: 49d08ae

Metrics [docs]

✅ unchanged

History

💚 Build #78871 succeeded 6a4bf73cafa59f17ea0aaee45fc636a494512c41

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

afharo

Yay!!! @TinaHeiligers thank you for fixing the tests!! It LGTM!

afharo · 2022-10-14T07:52:42Z

x-pack/test/saved_object_tagging/functional/tests/visualize_integration.ts

        await PageObjects.visualize.ensureSavePanelOpen();
-        await selectSavedObjectTags('tag-2');
-
+        await selectSavedObjectTags(...['myextratag']);


nit: do we need the spread?

Suggested change

await selectSavedObjectTags(...['myextratag']);

await selectSavedObjectTags('myextratag');

Co-authored-by: Kibana Machine <[email protected]> (cherry picked from commit 0b10d6c)

kibanamachine · 2022-10-14T13:56:14Z

💔 Some backports could not be created

Status	Branch	Result
❌	7.17	Backport failed because of merge conflicts
✅	8.5

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 142917

Questions ?

Please refer to the Backport tool documentation

Co-authored-by: Kibana Machine <[email protected]> (cherry picked from commit 0b10d6c) Co-authored-by: Christiane (Tina) Heiligers <[email protected]>

TinaHeiligers added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:Saved Object Tagging Saved Objects Tagging feature backport:all-open Backport to all branches that could still receive a release v8.6.0 labels Oct 6, 2022

TinaHeiligers requested a review from a team as a code owner October 6, 2022 20:33

TinaHeiligers added the release_note:skip Skip the PR/issue when compiling release notes label Oct 6, 2022

TinaHeiligers marked this pull request as draft October 6, 2022 23:17

TinaHeiligers added 3 commits October 13, 2022 09:52

Unskips visualize integration

6c91648

Creates visualizations as test fixutres

b4d5b03

Refactors tests, fixtures and listing table

6b8183e

TinaHeiligers force-pushed the kbn-89958-visualize-tagging-fail branch from 6a4bf73 to 6b8183e Compare October 13, 2022 22:53

TinaHeiligers commented Oct 13, 2022

View reviewed changes

TinaHeiligers marked this pull request as ready for review October 14, 2022 00:21

Merge branch 'main' into kbn-89958-visualize-tagging-fail

49d08ae

TinaHeiligers commented Oct 14, 2022

View reviewed changes

afharo approved these changes Oct 14, 2022

View reviewed changes

TinaHeiligers merged commit 0b10d6c into elastic:main Oct 14, 2022

TinaHeiligers deleted the kbn-89958-visualize-tagging-fail branch October 14, 2022 13:51

kibanamachine mentioned this pull request Oct 14, 2022

[8.5] Unskips visualize integration tests (#142917) #143375

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 14, 2022

Unskips visualize integration tests (elastic#142917)

63e6e86

Co-authored-by: Kibana Machine <[email protected]> (cherry picked from commit 0b10d6c)

TinaHeiligers added backport:skip This commit does not require backporting and removed backport:all-open Backport to all branches that could still receive a release labels Oct 14, 2022

kibanamachine added a commit that referenced this pull request Oct 14, 2022

Unskips visualize integration tests (#142917) (#143375)

18b5c89

Co-authored-by: Kibana Machine <[email protected]> (cherry picked from commit 0b10d6c) Co-authored-by: Christiane (Tina) Heiligers <[email protected]>

kibanamachine added the v8.5.0 label Oct 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unskips visualize integration tests #142917

Unskips visualize integration tests #142917

TinaHeiligers commented Oct 6, 2022 •

edited by kibanamachine

Loading

elasticmachine commented Oct 6, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers commented Oct 14, 2022

TinaHeiligers left a comment

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

TinaHeiligers Oct 13, 2022

kibana-ci commented Oct 14, 2022

afharo left a comment

afharo Oct 14, 2022

kibanamachine commented Oct 14, 2022

	await selectSavedObjectTags(...['myextratag']);
	await selectSavedObjectTags('myextratag');

Unskips visualize integration tests #142917

Unskips visualize integration tests #142917

Conversation

TinaHeiligers commented Oct 6, 2022 • edited by kibanamachine Loading

elasticmachine commented Oct 6, 2022

Choose a reason for hiding this comment

TinaHeiligers commented Oct 14, 2022

TinaHeiligers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Oct 14, 2022

💚 Build Succeeded

Metrics [docs]

History

afharo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Oct 14, 2022

💔 Some backports could not be created

Manual backport

Questions ?

TinaHeiligers commented Oct 6, 2022 •

edited by kibanamachine

Loading